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APPARATUS AND METHOD FOR CONGESTION CONTROL 
IN HIGH SPEED NETWORKS 

CROSS REFERENCE TO RELATED APPLICATIONS 
This application is a continuation of Application No. 08/922,457 filed 
5 September 3, 1997, which claimed the benefit of U.S. Provisional Application No. 
60/025,099 to Zhao et al., filed September 3, 1996, the disclosures of which are 
expressly incorporated herein by reference in their entireties. 

BACKGROUND OF TPIE INVENTION 
1. Field of the Invention 
10 The present invention relates to the field of communications, and to an 

apparatus and method for congestion control in high speed networks. More 
particularly, the present invention relates to flow control techniques and the flow 
control of Available Bit Rate (ABR) traffic in, for example, Asynchronous Transfer 
Mode (ATM) networks. 
15 2. Background and Material Information 

With advanced ATM technologies, future high speed networks will be able to 
support a wide spectrum of applications with diverse traffic characteristics and service 
requirements. Using ATM Forum terminologies, traffic is classified into three 
classes: Constant Bit Rate (CBR), Variable Bit Rate (VBR) and Available Bit Rate 
20 (ABR). The CBR and VBR traffic are transmitted in high priority with sufficient 
reservation of buffer space and link capacity; their arrival rate is normally not 
adaptable to the changing network environment. In contrast, the ABR traffic is 
transmitted in low priority using the remaining buffer space and link capacity; their 
arrival rate is constantly adjustable to avoid network congestion. For low complexity 
25 high speed operation, the end to end flow control approach is found superior to that 
of link by link flow control. Further, the traditional window based flow control is no 
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longer effective in high speed networks and is replaced by rate based control. In the 
rate based control design, the arrival rate of each ABR connection is dynamically 
adapted based on network feedback information. 

A simple rate based control scheme uses single bit feedback information from 
5 the network. An example of such a control scheme is described in M. HLUCHYJ and 
N. YIN, "On Closed-loop Rate Control for ATM Networks/' Proc. INFO-COM ^94 . 
pp.99-108 (1994), the disclosure of which is expressly incorporated herein by 
reference in its entirety. With single-bit feedback, a bit is marked at switching nodes 
or destination based on some preset congestion status. All the ABR sources will then 

10 adjust their transmission rate, based upon the single bit observation. An exemplary 
single-bit feedback scheme is called negative polarity feedback, and is described in 
J. C. BOLOT and A. SHANKAR, "Dynamic Behavior of Rate-based Flow Control 
Mechanisms," ACM Comp. Comm. Review , Vol. 20, No. 2, pp. 35-49 (1992), the 
disclosure of which is expressly incorporated herein by reference in its entirety. In 

15 negative polarity feedback, each ABR source reduces its transmission rate 
exponentially once receiving the negative feedback bit, otherwise each source keeps 
increasing its transmission rate linearly. 

As indicated in the study described in R. JAIN et ah, "The OSU Scheme for 
Congestion Avoidance Using Explicit Rate Indication," OSU Technical Report . Sept. 

20 1994, the disclosure of which is expressly incorporated herein by reference in its 
entirety, a single-bit rate based control is too slow to react to the rapidly changing 
traffic environment and an explicit rate control scheme should be designed to enhance 
control performance. One scheme was proposed by JAIN and his co-workers at Ohio 
State University. In the OSU scheme, a load factor is periodically computed at the 

25 congested node, the load factor being the ratio of present aggregate ABR rate to its 
desired rate. The rate of each individual ABR source in the next period is then simply 
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equal to its current rate divided by the load factor. For example, if the aggregate rate 
is only one half of its desired rate, each source rate will be doubled in the next period. 

The major advantage of most proposed single bit and explicit rate control 
schemes is their simplicity in implementation. However, to achieve such simplicity 
5 neither a dynamic control model nor round trip multiloop delays are considered in the 
above noted schemes. Yet, there is a price for simplicity which should be considered. 
First, most existing control schemes induce significant low frequency high magnitude 
oscillations in each feedback ABR control loop, where the oscillation frequency is 
directly associated with the round trip delay of each loop. It is intuitively clear that 

10 such inherent low frequency traffic oscillations will cause substantial oscillations in 
queuing process. As a result, a large buffer capacity has to be reserved at each node 
to absorb the ABR traffic oscillations. Second, the stability of the control schemes 
is highly sensitive to the high priority CBRA^BR traffic characteristics. 

Another problem with most existing attempts on ABR feedback control design 

15 is they generally neglect the effect of multiple feedback loop delays and high priority 
traffic transmission. With multiloop delays, the already existing low frequency high 
magnitude oscillations of ABR traffic within each loop can become much worse. 
With high priority traffic transmission, the original control stability conditions can be 
ruined. 

20 The closed loop stability problem is a major issue for any feedback control 

scheme. Its solution requires the knowledge of round trip delays of all ABR 
cormections. In fact, the round trip delay is the main obstacle that prevents the prior 
systems from achieving good congestion control performance; especially within a 
wide area network, the round trip delay of individual ABR connections can be vastly 

25 different. For systems with multiple delays, their feedback controller must be 
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carefully designed to provide closed loop stability, otherwise the systems can easily 
become unstable. 

The study in L. BENMOHAMED and S. MEERKOV, "Feedback Control of 
Congestion in Store-and-Fonvard Networks: The Described Case in Single 
5 Congestion Node," IEEE/ACM Trans. Networking , Vol. 1, No. 6, Dec. 1993, pp. 693- 
708, the disclosure of which is expressly incorporated herein by reference in its 
entirety, considers round trip delays and analyzes the closed loop stability of an 
explicit rate control scheme in the absence of high priority traffic. In the disclosed 
scheme, a fluid flow queuing model is adopted and the queue size at the bottleneck 

10 node is used as the feedback information for the ABR traffic adaption. The feedback 
controller in BENMOHAMED et al. is designed only with the requirement of closed 
loop stability. No other performance criteria such as steady state error is considered. 

hi existing binary bit Available Bit Rate (ABR) congestion control schemes, 
nodal congestion is often detected by comparison of present queue size with a 

15 predetermined threshold. However, the queue threshold detection schemes have 
disadvantages. First, no congestion can be detected until the queue is longer than the 
threshold. Second, although the congestion is readily removed as the queue starts 
dropping, the detection of when the queue dropped below the threshold has 
unnecessarily extended the congestion period. The delayed detection followed by 

20 unnecessary extension of congestion periods significantly increases the oscillation 
period as well as the oscillation magnitude of ABR traffic within the network, causmg 
a large consumption of buffer resources. 

Therefore, despite the advances, congestion control of high speed network 
switches, such as ATM switches, is still inadequate. Large buffers are required to 

25 handle the overflow of data due to inaccurate detection of the network congestion. 

Moreover, multiple feedback loop delays and high priority traffic transmission has 
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been neglected in existing feedback control designs. Thus, there exists a need for a 
system which accurately detects the congestion in an ATM network in order to 
optimize the throughput of ATM networks by controlling the flow of the ABR traffic. 

SUMMARY OF THE INVENTION 
5 In view of the foregoing, the present invention, through one or more of its 

various aspects, embodiments and/or specific features or subcomponents, is thus 
intended to bring about one or more of the objects and advantages as specifically 
noted below. 

A general object of the present invention is to provide an apparatus and method 
10 for congestion control of high speed network switches, including ATM switches. 

A further object of the invention is to provide a system which is capable of 
accurately detecting congestion of high speed network switches, such as ATM 
switches, in order to optimize the throughput of the network by controlling the flow 
of traffic. 

15 According to an aspect of the invention, the link capacity requirement of 

CBRA^BR traffic is captured by its present filtered rate so that the ABR transmission 
rate will be periodically adapted to the remaining link capacity. However, because 
of multiloop delay and the CBRA/^BR traffic variation, the link capacity will not be 
fully utilized. Another object of the present invention is therefore to minimize the 

20 unused link capacity. An advantage of the present invention is it eliminates 
unnecessary highly frequent adaptations of ABR traffic rate as found in most existing 
control schemes, which are caused either by the high frequency variation of 
CBRA^BR traffic or by the rapid change of queue congestion status. Because of the 
filtering, the ABR traffic varies very smoothly along with the slow time variation of 

25 the low frequency CBR/VBR traffic. 
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Another object of the invention is to provide an explicit rate ABR traffic 
comtrol scheme by using a process control design method, referred to herein as 
Generated Prediction Control (GPC), to provide a closed loop stable controller with 
linear quadratic optimal performance. An advantage of the present invention is that 
multiloop delays of ABR connections are built into the design of the control model. 
The low frequency high magnitude oscillations of ABR traffic flow, as usually found 
in most proposed ABR control schemes, are therefore eliminated. As a result, not 
only is the buffer capacity requirement for ABR traffic substantially reduced, but the 
control stability condition is also significantly improved. 

In a preferred embodiment, a new explicit rate control scheme based on the 
results of frequency domain analysis of multimedia traffic is provided. Examples of 
frequency domain analysis of multimedia traffic can be found in S. Q. LI and C. 
HWANG, "Queue Response to Input Correlation Functions: Continuous Spectral 
Analysis," IEEE/ACM Trans. Networking , Vol. 1, No. 6, Dec. 1993, pp. 678-692 (LI 
(I) et al. hereinafter), and S. Q. LI, S. CHONG, and C. HWANG, "Link Capacity 
Allocation and Network Control by Filtered Input Rate in High Speed Networks," 
IEEE/ACM Trans. Networking , Vol. 3, No. 1, Feb. 1995, pp. 10-15 (LI (II) et al. 
hereinafter), the disclosures of which are expressly incorporated herein by reference 
in their entireties. The inventors have found that the link capacity required by input 
traffic at each node is essentially captured by the traffic's low frequency 
characteristics. In other words, no congestion occurs at the node if the control design 
guarantees that the aggregate CBR/VBR/ABR traffic rate, filtered in a low frequency 
band, never exceeds the link capacity. All the high frequency traffic is absorbed and 
smoothed out via limited buffering. Further, the filtered traffic rate is significantly 
less than its original non-filtered rate. 
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Another object of the present invention is to provide significant improvement 
of ABR traffic performance by replacing the queue threshold detection with the 
bandwidth threshold detection, using an Explicit Forward Congestion Indication 
(EFCI) scheme. According to an aspect of the invention, a modified EFCI scheme is 
5 provided, which is called EFCI-ECD scheme, where ECD refers to early congestion 
detection. A simulation study has shown about a 60% savings in buffer capacity for 
the EFCI-ECD scheme to achieve the same throughput as the original EFCI scheme, 
providing the same set of source control parameters. The amplitude of the ABR 
traffic oscillation is reduced by about 50%. Note that the traffic filtering operation 
10 can easily by implemented by digital signal processing (DSP) chips. Using today's 
technology, a common DSP chip only costs a few U.S. dollars whereas the high speed 
SRAM chips for buffer capacity are relatively much more expensive. Moreover, a 
single DSP chip can be shared by many links for multiple traffic measurement 
purposes. 

15 Thus, there are at least three major advantages for the EFCI-ECD scheme of 

the present invention: (1) none of the existing EFCI protocols at source and 
destination needs to be changed; (2) the buffer capacity required at each switching 
node is substantially reduced; and (3) the ABR traffic oscillafion is much reduced. 
The present invention is described as a traffic congestion control apparatus for 

20 use in a network having a plurality of types of data traffic. The data traffic comprises 
high priority traffic and low priority traffic. The network has a plurality of links 
through which the data traffic flows, each link being susceptible to data traffic 
congestion. The apparatus has a processor which comprises a filter through which 
said data traffic flows, and a sampler which periodically measures a characteristic of 

25 the data traffic filtered by said filter. The measured characteristic indicates a present 
link capacity requirement of the filtered traffic. A flow control system for adjusting 
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a flow rate of the low priority traffic is also provided. The apparatus also comprises 
a comparator which compares the measured characteristic with a predetermined 
threshold. The comparator sends a first signal to the flow control system when the 
measured characteristic exceeds the predetermined threshold to indicate link 
5 congestion. The comparator sends a second signal to the flow control system when 
the measured characteristic is below the predetermined threshold to indicate unused 
link capacity. The flow control system reduces the transmission rate of the low 
priority traffic in response to receipt of the first signal and increases the transmission 
rate of the low priority traffic in response to receipt of the second signal. 

10 The traffic congestion control apparatus may be used in an asynchronous 

transfer method (ATM) network. 

In a preferred embodiment, the filter is a low pass filter. 
In another embodiment, a method for controlling congestion in a network is 
provided. The network has high priority data traffic and low priority data traffic, the 

15 low priority traffic being transmitted by a plurality of data inputs. The network 
comprises a plurality of links through which the data traffic flows. The method 
comprises filtering the data traffic and then periodically repeating the following steps: 
sampling the filtered traffic to estimate a present link capacity required by the high 
priority traffic; determining the remaining link capacity; calculating a transmission 

20 rate for each data input to minimize an unused link capacity of each link subject to no 
congestion; and adjusting a transmission flow rate of each data input to match the 
calculated transmission rate. 

The above listed and other objects, features of advantages of the present 
invention will be more fully set forth below. 

25 BRIEF DESCRIPTION OF THE DRAWINGS 
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The present invention is fiirther described in the detailed description which 
follows, by reference to the noted plurality of drawings by way of non-limiting 
examples of preferred embodiments of the present invention, in which like reference 
numerals represent^^ilar parts throughout the illustrations, and wherein: 

is a block diagram of an exemplary ATM switch, with input buffer 
swtcn architecture, according to an aspect of the present invention; 

are plots of a 28-sec JPEG video sample sequence and the 
same z^ec JPEG vide.o^sample sequence as filtered by non-causal low-pass filtering; 

illustrate ABR transmission rates without filtering delay, 
fflustrates the first 5 groups, and Fig. 3(b) illustrates the second 5 
10 groups of video sources; 

4(b) are graphs of buffer occupancy and unused link capacity; 
of a filtered 28-sec JPEG video sequence which has been 
filtered^ causal low pass filtering; 

6(a) a^d-6(b) illustrate the ABR transmission rate with a 70ms filtering 
delay ofjJ?>^irst 5^0ups and the second 5 groups of 10 groups of video sources; 

7(b) are graphs of buffer occupancy and remaining link capacity 
witti/nitering dtlsy^^ 

ffof buffer occupancy at 80% link capacity utilization; 
9(b) illustrate the ABR transmission rates for the first 5 groups 
econd 5 groups of 10 groups of video sources with adaptation to balancing 
of tjfe ABR connections; 

Fig. 10 k-af^aph of buffer occupancy behavior with random round trip delays 
at 99% linkxapacity utilization; 

/Fig. 1 1 is^^^faph of buffer occupancy behavior with aggregation of ten high 
privity JPEG sources with filtering; 
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Fi&,/}^is a graph of buffer occupancy behavior with aggregation of ten high 
priopty MPEG soujpe^with filtering; 

Fig JiS^l^^plf^fb^ occupancy capacity for ten JPEG without filtering; 
^Fig^^i^Hfa graph of buffer occupancy capacity in presence of ten MPEG video 
sources^ithout filtering;.>'^'"^ 
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Fig. 15 is-^a graph of aggregate ABR traffic showing a comparison between the 
GPC method of the present,invention and the load factor approach of the prior art; 

^ig. 16i^:?-tf^^^ph of a single group of ABR traffic in the presence of high 
priority te^mc using t^^li^ad^factor approach of the prior art; 

^ Fig. n^is^Cplot of congestion detection comparing the bandwidth threshold 
schem^OTthe present invention and a queue threshold scheme of the prior art; 

^.^;:^^^^ig IS^is^lofcCd^ of an exemplary network architecture, with a single 
congestiomliSk and ten ABR connection groups, in accordance with an aspect of the 



present invention; 

Figs. l^^S^and 19(b), respectively, are plots of buffer occupancy of the EFCI 
and EFPMCD schemes for the 10 segment MPEG case; 

Figs. 20(^Mfa 20(b), respectively, are plots of buffer occupancy of the EFCI 
and EFCI-g€!D schemes^for the 100 segment MPEG case; 



Fflg. 21 is^Cplot of the>-aggregated ABR traffic rate for the EFCI-ECD and 
EFCI schemes ow>a period of one second; 



'ij^2 is a plot showing the effect of the queue length at time t on 
maximum/average::queue for the EFCI scheme; 

Fig.y:23 is a plot showing the comparison of maximum/average queue at 
different utilization^rates; 

r T:,rr nA^\^ a comparison of maximum/average queue for different additive 
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¥\g.J}^\s a block diagram of an exemplary feedback control design of the 
inventii 

DETAILED DESCRIPTION OF THE INVENTION 
Referring now to the accompanying drawings, Fig. 1 illustrates a block 
5 diagram of an asynchronous transfer method (ATM) switch with input buffer switch 
architecture. Although actual implementation of the explicit forward congestion 
indicator, early congest detection (EFCI-ECD) scheme of the present invention 
depends on the type of ATM switch fabric, for purposes of illustration, the features 
of the invention will be described with reference to an input buffer switch 
10 architecture. Of course, the same principles of the invention will apply for an output 
buffer switch architecture or a shared buffer switch architecture. Further, although 
W the various features of the invention are described herein with reference to ATM 

^0 switches, the various features and aspects of the invention may also be applied to 

m other types of high speed network switches. 

h 15 Fig. 1 shows a block diagram of an ATM switch 10 utilizing the EFCI-ECD 

j^n scheme of the present invention. Incoming ATM cells arrive at each input port 1, and 

I;!: then are temporarily stored in an input buffer 2. Each input port 1 has one input or 

^=3 queue buffer 2 to receive the incoming ATM cells. A switch medium 3 directs the 

ATM cells to an appropriate output port 5 based on a predetermined routing derived 
20 from the header information contained within a particular ATM cell to be routed. The 

switch medium 3 may be, for example, a cross bar, or TDM (Time Division 

Multiplexing) bus and the queue buffer 2 may comprise a SRAM or other memory 

component. 

A processor 4 is provided, which may comprise one or more DSPs (digital 
25 signal processors) and/or general purpose processors. The DSPs and/or general 
purpose processors of the processor 4 may be any suitable combination or set of well 
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known DSPs or general purpose processors, such as those available from Texas 
Instruments, Analog Devices or Motorola. Where one DSP is provided, each of the 
input ports 1 is controlled by a single processor 4. If multiple DSPs are provided, 
each of the input ports 1 may be controlled by its own dedicated processor 4, In the 
5 multiple DSP configuration, a master controller (not shown) may be provided for 
coordination, control and synchronization of each of the DSPs. The processor 4 is 
programmed to execute the EFCI-ECD method of the invention, as described below. 
The functions of the processor 4, however, are not limited to performing the described 
EFCI-ECD scheme, and may also include other functions such as process call setup, 

10 and policing input traffic, etc. Further, although a DSP has been described as 
performing the functions of the processor 4, an application specific integrated circuit 
(ASIC) may be used in place of a DSP, which could be designed specifically to 
execute, for example, the low pass filtering of the EFCI-ECD scheme. 

To explain the operation of the ATM switch 10, the processing for one input 

15 port 1 will now be described. The processor 4 counts the number of arrival ATM 
cells from each virtual channel of the input port 1 for a time interval T. Dividing the 
cell arrival counts by the time interval T yields the input arrival rate of the virtual 
channel. The arrival rates of all virtual channels of the input port 1 is calculated by 
the processor 4 at each time interval T. 

20 A pre-designed low pass fihering function may be stored in the processor 4. 

Further, the processor 4 may be programmed by well known techniques, i.e., storing 
the denominator and numerator coefficients in a ROM or RAM, and downloading the 
coefficients on power up. By inputting the arrival rates of all CBR^VBR/ABR virtual 
channels to the low pass filtering flinction of the processor 4, filtered CBIWBR/ABR 

25 traffic arrival rates are obtained. 
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To detect a congestion condition at the input port 1, a total of the filtered 
CBRAHBR/ABR virtual channel traffic arrival rates is calculated by the processor 4, 
and if the total rate is greater than the link speed of the input port, the link is 
determined to be congested. Otherwise, the link is determined not to be congested. 
5 Alternatively, filtered CBRA^BR arrival rates may be used to determine if the link is 
congested. 

If the link is determined to be congested, the EFCI bit of the ABR data cell is 
set to indicate the congestion condition. Once the EFCI bit is set, the congestion 
condition can be alleviated using well known techniques. 
10 1 . DYNAMIC MODEL OF THE CONGESTED NODE 

In this section, network parameters are defined and the dynamic equation of 
the present invention is derived. The object is to minimize the unused link capacity 
in steady state subject to no congestion. 
1 . 1 Network Parameters 
15 Assume a single bottleneck link L within the network. There are M number 

of ABR connections via link L. Denote the set of M source-destination pairs by 

S = {(s,d),...(s,dU 
Introduce the following notations for further discussion: 
r(t): aggregate high priority CBR/VBR traffic arrival rate in link L at time t; 
20 Ui,(t): ABR traffic transmission rate from the source destination pair (s,d)k at time t; 
control update period; 
\\ round trip delay of the k's source-destination pair; 
x(t): unused link capacity. 

In the congestion control apparatus and method of the present invention, Uk(t), Vk, are 
25 the input variables, r(t) is the external disturbance, and x(t) is the controlled variable. 
The objective is to minimize x(t) in steady state subject to no congestion. 
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The round trip delay t,, consists of the following components: the packetization 
and depacketization delay, propagation delay, switch processing delay and queuing 
delay. As described in M. D. PRICKER, "Asynchronous Transfer Mode — Solution 
for the B-ISDN, Ellis Horvvood Press (1993), the disclosure of which is expressly 
incorporated herein by reference in its entirety, only the queuing delay is random, 
which is expected to be less than ten percent of the overall round trip delay in wide 
area networks. For simplicity, the control design assumes that each ABR connection 
has a fixed round trip delay. 

In every t^ second, the controller at node L estimates the link capacity 
requirement of the present r(t) via the low pass filter operation. The remaining link 
capacity is then properly divided among all ABR connections by the congestion 
control algorithm. For practical control implementation, each round trip delay needs 
to be quantized in units of t^: 



where represents the quantization error and represents the number of connections 
in group \ (described below). By approximation, 



Further, the number of ABR connections at each node in wide area networks, M, 
tends to be very large. Since M is the size of the input vector, a large input gain 
matrix may be obtained. The control solution with large M may not be feasible in real 
time operation. One alternative approach is to partition S into subsets by the round 
trip delay grouping. That is, all connections with round trip delay equal to kt^ will be 
grouped into subset \. There are N such subsets: 





jt = l,,..,M, 



^ nj^xt,, fork=l,...,M. 
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8)5 -..J Sj^. 

In practice, N may not be very large. For example, taking N = 10 allows the 
maximum round trip delay equal to lOt^. If t^ = 7 ms, the maximum round-trip delay 
70ms is longer than the United States coast to coast round trip propagation delay. The 
5 aggregated ABR transmission rate in each subset S,, is denoted by The grouped 
input vector is therefore described by: 

U- [Uj, Uk, Un^- 

Once u is computed by the control algorithm, its individual u,, will be evenly divided 
among all the ABR connections in S,,. Such a division within each subset S^ is 
10 completely separate from the computation of u; its execution only requires the present 
number of ABR connections in Si,. 

1 .2 Congestion Control Model 

Because of the discrete time control operation, all system variables r(t), u^(t) 
and x(t) are changed into r(n), Ui,(n) and x(n) in units of t^,, respectively. The system 
1 5 dynamic equation can then be expressed by 



^ x(« + l) = x(n)^(r(n)-r(n + l)) + b^(u^(n-n^-l)-Uj^(n-n^)), 



where r(n) - r(n + 1) represents the one-step variation of high-priority traffic and u^(n - 
n,, - 1) - Ui,(n - n,,) is the one-step variation of ABR traffic. Note that % in (1) takes 
account of the n^-step loop delay on the k-th connection group. Further define b^ = 
20 0 for empty subset S^, and b,, ^ 1 otherwise. In wide area networks, each subset is 
expected to contain a certain number of active ABR connections. The chances for S^ 
to become empty are therefore negligible. One can generally assume 

bi-... = bk = ...=bN^l. 
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In the system disclosed herein, the controller will keep track of the number of ABR 
coimections in each subset S,^. For fairness, the rate assigned to each subset should 
be proportional to the number of its connections. For the sake of simplicity, assume 
the fairness to be achieved is given by 
5 u,(n) = U2(n) ... = u^Cn). 

The fairness problem will be discussed in further detail below. 

To convert the dynamic equation (1) to its Z-domain expression, define 
A(z-') = 1 - z\ 
u = [u„ ... Un^ , 
10 B(z-') = [b,(z-'), ... b^Cz-^)], 

bk(z-')=z-V'-l),Vk. (2) 
Applying them to (1) results in: 

A(z-')x(n+l)=B(z-')u(n)+(z-' - l)r(n+l), (3) 

For control purposes, 

15 x(n+l)=P(z-')u(n)-r(n+l), (4) 

r 

^ where P(z — - is the transfer function of the controlled system. 

^ J 

The control input u(n) must be non-negative, i.e., U|,(n) > 0, Vk. Since r(t) is 
highly unpredictable, r(n) in the system design is treated as an extemal disturbance 
with two main advantages. One is for simplicity of the control model whose 
20 complexity order is independent of the disturbance. The other is for stability of the 
control model where the stability condition is also independent of the disturbance. 

The frequency domain queuing analysis in LI (II) et al. indicates that the 
momentary link capacity requirement of r(t) is essentially captured by its filtered low 
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frequency rate denoted by r, (t). In other words, it is the low frequency behavior of 
r(t) that can possibly drive the nodal congestion; the high frequency portion of r(t) is 
perfectly transmitted using limited buffering. For effective congestion control, 
replace r(n) by rL(n) in the dynamic equation (1): 



N 

where r^Cn) is the filtered r(n) in a properly selected low frequency band. A detailed 
discussion of the cut off frequency selection for filtering can be found in LI (II) et al, 
and J. D. PRUNESICI and S. Q. LI, "The Linearity of Low Frequency Traffic Flow: 
An Intrinsic I/O Property in Queuing System," Proc. IEEE Infocom '95 Conference . 

10 April 1995, pp. 613-623,, the disclosure of which is expressly incorporated herein by 
reference in its entirety. Therefore, XL(n) corresponds to the unused low frequency 
link capacity. Similar to (4), the low frequency dynamic equation in the Z-domain is: 

XL(n+l) = P(z-Vn)-rL(n+l). (6) 
The difference between the original dynamic model (4) and the low frequency one (6) 

15 is given by: 

x(n) - XL(n) - r(n) - r^Cn), 



where r(n) - r^(n) represents the high frequency portion of r(n). Since the transport 
of high frequency traffic does not require extra link capacity, the two models are 
basically equivalent except that the control of u(n) by XL(n) is much simpler and more 
20 effective than that by x(n) as described in the following. 

The major advantages of using r^(n) are high stability, high efficiency and low 
complexity. Without filtering, the high-frequency variation of r(n) could cause the 
unnecessary high frequency fluctuations in the input rate u(n). As a result, all the 
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ABR sources would have to be designed with complexity for frequent adaptations of 
their transmission rate, which after all is ineffective to reduce the nodal congestion. 
Further, the high frequency variation of r(n) would easily mislead the direction of link 
capacity adaptation, which can cause a substantial reduction of link efficiency and 
5 even destroy the whole stability condition as explained below. Similar problems exist 
when the queue length at node L is chosen to be the controlled variable as in BOLOT 
et al,, and BENMOHAMED et al. Since the queue length is not linearly related to the 
link capacity assignment, it is always difficult to design a link capacity allocation 
algorithm without inducing the unnecessary ABR rate fluctuations. In contrast, 
10 because of the filter operation in the present invention, all the ABR transmission rates 
change smoothly along with the slow time variation of the low frequency high priority 
traffic. 

Accordinglv, one can avoid congestion by ensuring the low frequency link 

capacity constraint ^u^{n)+r^{n) < C, Vk, where C represents the link capacity 

J 

15 at node L. The control objective is to minimize XL(n) is steady state subject to no 
congestion. 



(Blank) 
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2. FEEDBACK CONTROL DESIGN 

The feedback loop control is achieved through the adaptation of the ABR 
transmission rate u(n) to the unused link capacity XL(n). In other words, a feedback 
controller design K(z ') is need such that the ABR input rate vector u(n) is adapted by 

u(n) = -K(z-')xL(n). (7) 
Fig. 25 illustrates, in block diagram form, an exemplary feedback control 
design of the invention, where the notations, u, x, Xl and r are used to represent the 
Z-transform of u(n), x(n), XL(n) and r(n). Substituting u(n) into (4), 

XL(n+l)=-P(z-')K(z-')xL(n)-r,(n+l), 
Let F(z"') be the low pass filter of the high priority traffic, 

r,(n)-F(z')r(n). 

. Let 

O 

5(z-)- ^ 



i+p(z-')is:(z-') 



Resulting in: 

u(n) - -K(z ^)S(z-^)r,(n), (8) 
Hence, the ABR transmission rate under this feedback control law contains no high 
frequency component. 

Notice that many control schemes can be applied here for the solution K. In 
accordance with an aspect of the invention, a Generalized Predictive Control (GPC) 
scheme may be utilized for its simplicity in real time implementations and flexibility 
for systems with different orders. A GPC method was proposed in CLARK et al. and 
it has been applied to real chemical engineering control problems. An example of a 
GPC method, as applied to chemical engineering control, is described in D. W. 
CLARK, C. MOTHADI and P. S. TUFFS, "Generalized Predictive Control Part I, 
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The Basic Algorithm;* Automatica , Vol. 23, No. 2, pp. 137-148 (1987), the disclosure 
of which is expressly incorporated herein by reference in its entirety. 

The GPC method of the present invention may be used to solve the ABR 
feedback control problem. There is a close relation between GPC and the Linear 
5 Quadratic Regulation (LQR) problem. A brief description of the GPC method 
follows. 

A preferred technique for designing the GPC controller is to solve the so called 
Diophantine equation which is in polynomial form: 

Ek(z-OA(z-')A(z 0+Fk(z-^)z-'^= 1 (9) 

10 where k is an integer. A is the original control system descriptor, which in this case 
is given by A(z"0 = 1 - z"' defined in (2). The integral function a(z'^) = 1 - z"^ is 
embedded in the controller to ensure the zero-error performance in steady state, i.e., 
lim^^^ xjn) ^ 0, when rL(n+l) - rL(n) becomes a step function (the worst case extemal 
disturbance). and are the polynomial solution at each given k. There are many 

15 ways to solve (9). One efficient algorithm can be found in CLARK et al. The 
solution Fj, and are related to A and k, independent of the system variables Xl, u 
and r^, and can be obtained by offline computation. 

Multiplying XL(n+k) on both sides of (9) and then replacing A(z"^)xL(n+k) by 
B(z"')u(n+k - 1) from (3), we readily obtain 

20 XL(n+k)-E;,(z-OA(z')B(z^)u(n+k-l)+Fk(z-^)x^(n). (10) 

Hence, given the current unused link capacity x^{nX its k-step ahead value, XL(n+k), 
can be computed from the (k-1) step ahead input rate u(n+k-l). Taking k = 1, H 
for a total of H steps, define 
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e R 



H 



(11) 



U{n) = 



u(n) 
u(n +H-1) 



and further introduce a weighting matrix A 



A = 



1 
2 

H 



X 



dg 



E R 



HNxHN 



(12) 



(13) 



5 



where N is the number of source destination groups S. A.^, which must be positive, is 
the penalty factor for the adaptation of the input rate Ui^, Vk. 
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Define the optimal criterion by 

J[Uin)] - £{n-^\)X^{n^\)^U\n)KU{n). 

^ (14) 

where 

r 

X^(n -^\)Xj(n + \) represents the accumulation of squared unused link capacities in 

5 consecutive H steps. As in most control analyses^ the primary reason to include 
U^(n)AU(n) in (14) is to ensure the non-singularity of the control problem. It is also 
used to achieve the fairness for link capacity allocation among the N groups of ABR 
connections, through the weighting matrix A. Increasing penalty factor has the 
effect of reducing the k-th group transmission rate Uj,. 

10 The control problem is therefore formulated as to find the solution U(n) for the 

minimization of J[U(n)] at each n. Although U(n) contains H step control inputs 
[n(n), u(n+H-l)], only the most current one u(n) is used. From the solution u(n) 
and the present disturbance rL(n+l), the system dynamic equation (6) is used to 
compute the next step output XL(n+l). Recursively, fi-om XL(n+l) the above control 

15 procedure is used to compute the next step solution u(n+l). 

Here the basic steps for the computation of u(n) are briefly outlined. Refer to 
CLARK et al. for the detailed computation procedure. Rewrite (10) in terms of 
XL(n+l) and U(n) after taking the H consecutive steps: 

X(n-i-l)-GU(n)+f(n) (15) 

20 where G is a lower triangular matrix with G e R^"''^". From (10) the matrix G is 
expressed fi^om the combination of and F,^, Vk. Similarly, the vector f(n) can be 
expressed as a function of u(n-l) and XL(m) for some m < n. Substituting (15) into 
(14), results in: 
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= 2g^(G [/(«)+/(«)) +2 AC/(«). 



dU (16) 



The optimal solution U„_, is derived at — = 0 : 



While the matrix G can be singular, it is the weighting matrix A that ensures the non- 
_singularity of (A+G^ G). The one step input vector u(n) is then obtained by 



I 5 = -^fi""-'-'"' (.7) 



where Q, = [1^, 0, ...0] e R^"^" takes the first component of U(n) and a(z^)=1-z^ is the 
integral embedded in the controller. Note that an equivalent closed form of the 
feedback controller K(z"^) can be found in a similar way. The only computational 
complexity of the control is to derive the inverse matrix [A+G^G]"^ Since G is fixed 

10 for a given control system, the inverse matrix is computed each time when A is meant 
to change. This occurs when the number of ABR connections in each group is 
significantly changed, so that a new A is required to achieve the faimess. In practice, 
the number of ABR connections can be requantified in each group and a set of 
possible As can be designed. The solution [A+G^G]"^G can then be computed in 

15 advance and stored in the system for real time operation. 

The fairness performance is an important issue for traffic management. Recall 
that the transmission rate of each individual ABR connection is assigned at two levels. 
At the first level, the total available link capacity is properly divided among [U|,U2, 

23 



P15868.S01 

%], where the relative magnitude of should be proportional to the number of ABR 
connections in the k-th group. At the second level, u,, is evenly divided among all the 
ABR connections of the group. With the GPC control scheme, the relative magnitude 
of u,,'s can be tuned by adjusting the relative value of the penalty factor A \, 
5 Regrettably, there is no direct quantitative relation between and \x^. The design of 
is not only related to the numbers of ABR connections among the N groups, but 
also dependent on their round trip delay difference. In practice, some rules of thumb 

vp^ may be observed. For example, one can set ^ 2 to achieve % ^ Uj^+j, given 



that the round trip delay is approximated by kt^ and (k+l)t(, with respect to the k-th and 
10 (k+l)-th groups. The off line trial and error approach can always be adopted here by 

adjusting X^jg till the fairness of u is obtained. The selection of A^g is independent of 

the high-priority traffic. 

3 . EFCI-ECD CONTROL SCHEME 

It is commonly known that the current EFCI scheme takes large buffer 
15 resources with severe oscillation of the ABR traffic. As seen in the next subsection, 

these two key phenomena are closely related to the congestion detection mechanism 

used by the EFCI scheme. 

3 . 1 Early Congestion Detection 

As mentioned above, nodal congestion occurs whenever the filtered rate of 
20 aggregate traffic exceeds the link capacity. Using the queue threshold detection 
scheme, the detection of congestion occurrence has to be delayed by the amount of 
time required to build up the queue. Similarly, the detection of congestion resolution 
is also delayed by the amount of time required to bring down the queue. 
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The sample path of a queue and its associated filtered input rate is examined 
in Fig. 17. The input traffic consists of CBRA^R/ABR. The CBRA^R traffic are 
generated from 10 real time MPEG videos using Bellcore's coUecdon, which are 
transmitted with high priority, as described in M.W. GARRETT, and W. 
5 WILLINGER, "Analysis, Modeling and Generation of Self-Similar VBR Video 
Traffic," Proc. ACM Sigcomm , London, Sept. 1994, pp. 269-280, the disclosure of 
which is expressly incorporated herein by reference in its enfirety. The ABR traffic 
consists of ten connection groups, which are controlled by the EFCI scheme with 
queue threshold detection. Detailed configuration of the simulation is provided in the 

10 next subsection. The queue threshold is properly adjusted and set at 200 cells to 
achieve the target link utilization of 90%. As detected by the queue threshold in Fig. 
17, the first congestion period starts at t=0.527 sec and ends at t=0.681 sec, lasting for 
0.154 sec. In contrast, the bandwidth threshold detection scheme is used and the 
bandwidth threshold is set equal to 90% link capacity for comparison with the filtered 

15 input rate. As also shown in Fig. 17, the first congestion period starts at t=0.513 sec 
and ends at t=0.597 sec, lasting for 0.084 sec. For comparison purposes, the two 
thresholds in Fig. 17 are overlapped after properly scaling the filtered input rate on 
the left side Y-axis and the queue length on the right side Y-axis. This example 
clearly shows the shortcoming of the queue threshold scheme. 

20 It is not difficult to understand why the bandwidth threshold detection scheme 

significantly outperforms the queue threshold detection scheme. First, the early nodal 
congestion detection will lead to the early feedback control reaction by ABR sources 
to improve the performance. Second, the much reduced congestion period will 
significantly reduce the oscillation periods in each of the ABR control loops and 

25 hence result in much reduced oscillation magnitude. (The second aspect of the 
improvement plays a more important role than the first one.) These advantages of 
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bandwidth threshold detection are indeed verified by the following studies in the 
proposed EFCI-ECD scheme. 

4. SIMULATION STUDY OF FEEDBACK CONTROL OF MULTILOOP ABR 
TRAFFIC 

5 For the simulation study, refer to Fig. 18. A single congested node 20 within 

the network 10, supporting ten groups of ABR connections 12 {Sj, S2, Sjo), i.e., 
taking N = 10 in (1) is considered. The control updating period t^. = 7ms is chosen. 
The feedback loop delay of each connection in is fixed at kt^, unless otherwise 
stated. The multiple loop delays within the network are then in the range of 7ms to 

10 70ms, which is equivalent to one to ten control steps. 

The CBR/VBR high-priority traffic r(t) at the node 20 is generated from real 
JPEG/MPEG video segments of the movie Star Wars, which are highly bursty and 
strongly correlated, as described in GARRETT et al. The original JPEG video files 
were recorded in bytes in L4ms slice time units and the MPEG files were in 42ms 

15 frame time units. The bytes are converted into cells; each cell consists of 48 bytes of 
video and 5 bytes of header. In computer simulation, the queuing process is evolved 
at every 0.14ms in discrete time. Converting the input slice/frame time unit into the 
queuing simulation time unit, all cells generated in each slice/frame unit are assumed 
to be randomly allocated among the corresponding simulation time units by uniform 

20 distribution. 

The filtered high priority traffic is measured periodically at the control interval 
tc, denoted by r^(n) at the n-th interval. At each control step, the controller uses rL(n) 
and (8) to compute the optimal input rate vector u(n) = [Uj(n), U2(n), Uio(n)], which 
will feedback to each individual ABR source with its defined loop delay. In the GPC 

25 design 40 steps of prediction may be utilized, i.e. H = 40 in (1 1). 



26 



# 



P15868.S01 

The simulation study in the following subsections also considers the effect of 
random feedback loop delays and non-uniform ABR rate distributions. The total 
volume of the underlying high priority traffic changes from a single video source to 
the aggregation of 10 video sources. 
5 4. 1 With Non-Causal Filter 

One major issue in design of low frequency control is the selection of a low 
pass filter (LPF). A good LPF not only captures the low frequency characteristics but 
also requires less filtering delay. Intuitively, the filtering delay will postpone the 
network control response to burst arrivals of the high priority traffic. Note that the 
10 burst arrivals are typical low frequency behavior of the traffic. As a result, a longer 
filtering delay is more likely to mistrack burst arrivals which may cause nodal 
congestion. In practice, a prediction scheme to minimize the effect of the filtering 
delay in evaluation of Y^(n) may be designed. 

For simplicity, first consider a non-causal LPF which has no filtering delay. 
15 Although non-causal filters cannot be implemented in real systems, its control 
performance can be thought as the ideal case of the low frequency control. Also 
assume a fairness condition Uj ^ U2 ... ~ Ujo, which is approximately achieved by the 
following assignment: 

[X^,X2> ^10] = [150, 100, 80, 50, 30, 20, 10, 5, 2.5, I] (18) 

20 Note that \ declines inversely with its loop delay. 

The high priority traffic is represented by a 28-sec JPEG video sample path in 
Fig. 2(a). Its low frequency counterpart, filtered with a non-casual LPF at the cut off 
frequency a)^ = 45rad/sec, is plotted in Fig. 2(b) for comparison purposes. While the 
original peak arrival rate is 14Mbps, the filtered rate is only 8 Mbps. The average rate 

25 of the single video source is 3.0 Mbps. Assign the link capacity equal to C = 10 
Mbps, which is greater than the filtered peak rate of the video to ensure the delay 
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quality of the high priority traffic transmission. Figs. 3(a) and 3(b) illustrate the ABR 
rate time evolution of the ten groups, wherein Fig. 3(a) illustrates the first five groups 
and Fig. 3(b) illustrates the second five groups. Obviously, the ABR rate variation 
is caused by the low frequency variation of the high priority video. Unlike most 
5 proposed ABR control schemes, the ABR transmission rates using the new scheme 
contain neither high frequency variations, nor low frequency oscillations. The former 
has been removed by the filtering operation of the high priority traffic; the latter has 
been eliminated by the GPC method. 

In this example, the link capacity utilization is 30% for video and 69% for 

10 ABR traffic, which leads to the total utilization p=99%. Under such a high 
utilization, the aggregate queuing process in Fig. 4(a) is found surprisingly small, with 
respect to the maximum queue length qj^^^ = 95 cells. The advantages of low- 
frequency control and the GPC method are clearly seen. 

From Fig. 4(b) the behavior of the unused low frequency link capacity XL(t) can 

1 5 also be observed. For convenience, the solution x^(t) in Fig. 4(a) has been normalized 
by the total link capacity C. First, XL(t) converges rapidly from the initial worst case 
condition to zero. Second, the variation of XL(t) is insensitive to the bursty high 
priority traffic. Ideally, XL(t) =^ 0 is achieved. 
4.2 With Causal Filter 

20 Now the effect of filtering delay on the control performance is examined using 

the same example except the non-causal filter is replaced with a causal filter. Here 
a moving average LPF with hamming window is chosen. The window size is 
designed at 140ms; the filtering delay is equal to 70ms (one-half of the window size). 
Note that taking 140ms window size in the time domain filter design is somewhat 

25 equivalent to taking 45 rad/sec cut off frequency in the frequency domain filter 
design. For the discrete time implementation, each window consists of 100 steps with 
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1.4 ms per step. The filtered video sequence shown in Fig. 5 is basically identical to 
the one by the non-causal filter in Fig. 2(b) except with 70ms filtering delay. 

To identify the effect of filtering delay, assume the same link capacity C = 10 
Mbps as in the non-causal filtering case. The ABR transmission rate behavior of the 
5 ten groups in Figs. 6(a) and 6(b) (where Fig. 6(a) is the first five groups and Fig. 6(b) 
is the second five groups), is basically identical to that of the non-causal filter in Figs. 
3(a) and 3(b), which are smoothly adapted to the variation of the filtered high priority 
traffic. Similarly in Fig. 7(b), the unused link capacity basically stays at zero. As a 
result, the total link utilization also reaches 99%. The introduced filtering delay, 

10 however, requires much more buffer space as shown in Fig. 7(a), where q^^^ = 950 
cells in contrast to 95 cells without filtering delay. 

Note the rapid queue build up at the 17th sec, which corresponds to the worst 
burst arrival of the low frequency video in Fig. 5. Because of the 70ms filtering 
delay, the ABR rate controller will not be able to immediately respond to such a burst. 

15 As a result, the high priority burst arrival will cause the buffer accumulation for the 
consecutive 70ms period. From Fig. 5 it is seen that the low frequency burst jumps 
from 1 .7 Mbps to 8.5 Mbps, causing the queue increment at 5.8 Mbps rate for the next 
70ms. A simple calculation will show the equivalent queue built up equal to 958 cells 
(i.e., 5.8 Mbps x 70ms/53 bytes/8 bits), which is exactly what is seen in Fig. 7(a). 

20 Clearly, the filtering delay is inevitable with the low frequency control 

approach. The amount of the filtering delay is related to the low frequency 
characteristics of the high priority traffic and also to the selection of the LPF. In 
practice, a traffic prediction scheme may be applied to reduce the effect of the 
filtering delay. Once the filtering delay is fixed, the network must be designed with 

25 the capability to absorb the worst case low frequency burst arrivals for the filtering 
delay period. This can be achieved by the reservation of sufficient buffer space and/or 
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link capacity. The actual amount of the reservation can be quantitatively identified 
once the filtering delay and the allowable worst case burst arrivals are provided in the 
network design. 

For instance, more link capacity can be reserved instead of taking more buffer 
5 space to accommodate the filtering delay. In the previous example, the unused link 
capacity is targeted at zero for optimal control, which leads to p=99% and q^^^ = 950. 
On the other hand, the link utilization can be reduced for less buffer space. Change 
the target value of the unused link capacity from zero to 20% link capacity. 
Correspondingly, the design link utilization is changed from 100% to 80%. For the 
10 same example, q,^^^ is reduced to 180 cells at p=80% in Fig. 8. 
4.3 With Variation of ABR Connections 

In real networks, the number of ABR connections at each group is subject to 
change. For control fairness, the aggregate ABR transmission rate at each group, u^, 
should be adapted accordingly. Denote the number of connections in group by n^. 

15 For example, when n,, is reduced by 10%, the relative value of % in u should also be 
reduced by 10%. Under the GPC control, such faimess can approximately be 
achieved by tuning the associated penalty factor X^, So far no rigorous relation has 
been established between the penalty factor and the relative weight of u^ in u. 

Use the same 28-sec JPEG video sequence for the high priority traffic. For the 

20 first 5 ABR groups, suppose that % at k^l, 5 changes in every 2.8sec interval, 
consecutively for the whole 28 sec period. The changes in every 2.8 sec interval are 
represented by -3% for nj, +2% for n2, +3% for n3, -1% for n4 and -2% for n^, 
respectively. The positive sign in percent represents the relative increment of n^ and 
vice versa for the negative sign. The corresponding change of A,, at every 2.8 sec is 

25 assumed to be inversely proportional to the variation of n^. That is, +3% for Xy, -2% 
for -3% for A3, +1% for X^ and +2% for A5, respectively. The initial values of X^ 
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are assigned by (18) as in the previous examples for the uniform fairness: = n2 = 
= UiQ. The corresponding ABR traffic rate adaptations are shown in Figs. 9(a) and 
9(b). The comparison between Figs. 6(a) and 9(a) for the first 5 groups indicates the 
effectiveness of A^g adaptation for balancing the ABR connections. The main purpose 
5 of this example is to show the robustness of the proposed control algorithm to the 
changing environment of the ABR connections. 
4.4 With Random Round Trip Delays 

The analysis and simulation described so far have been based on the 
assumption of deterministic round trip delays, which is not realistic once the queuing 
10 process is involved. Here the effect of random delays on the control performance is 
examined. In high speed netAvorks, assume that about 10% of the round trip delay is 
attributed to the random queuing delay. Note that the control update period at the 
congested node is always fixed regardless of the randomness of round trip delays. 
The queuing process can cause delay variations of nodal feedback information to 
15 ABR sources and also delay variations of ABR traffic to the congested node. 
Express the round trip delay of the k-th connection group by 

t,(n) - kt, -H x,(n) (19) 
at the n-th control time interval t^. and t^is fixed at 7ms. x (fi) represents the 
accumulated random queuing delay of the k-th loop. Knowing that the queuing delay 
20 process should be strongly autocorrelated, the variation of x^(n) may be characterized 
as follows: 

x,(n+l) = 0.9x,(n)+b,G)(n), 
where (o(n) is a Gaussian white noise and the parameter is directly related to the 
variance of x,,. The coefficient 0.9 is chosen to reflect the strong correlation of 
25 queuing delays in adjacent time units. Also E[xi,(n)] = 0 because the average delay 
has already been taken into account by kt^ in (19). The distribution of x,, is Gaussian 
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in steady state. The parameter b,, is designed by the following tail distribution 
constraint: 

Pr(|x,(n)|<kms) = 0.98. 
Such a constraint design ensures that the random delay portion in each ABR 
5 connection is proportional to its average round trip delay. For instance, at k = 5 the 
average delay of t^(n) is fixed at 35ms and the random delays X5(n) fall in the range 
of [-5ms, +5ms] with 98 percentile. 

Using the above random delay model, the simulation study based on the same 
example as in Section 4.2 shows virtually no impact of the random delays on the 
10 queuing process (by comparison of Fig. 10 and Fig. 7(a)). In general, as long as the 
random delays are kept in a small portion of their round trip delays, the impact of the 
random delays on control performance is expected to be small. This is because the 
ABR transmission rates are smoothly adapted to the slow time variations of the low 
frequency high priority traffic. The time scale of such slow time variations is 
15 expected to be significantly longer than the round trip delays. Hence, the relatively 
small time varying scales of the round trip random delays should not have significant 
impact on the overall control performance. 

4.5 With Aggregate High Priority Traffic 

In all of the above examples, a single JPEG video source represents the 
20 underlying high priority traffic. In reality, a large number of CBRA^BR sources are 
likely to be multiplexed on a high speed link. By the law of large numbers, the 
aggregate high priority traffic is expected to become less bursty, which will improve 
the ABR control performance. 

For instance, randomly select ten independent 2 minute JPEG video sequences 
25 from the movie Star Wars. The average of the aggregate high priority traffic is 5 1 
Mbps. The link capacity C = 67 Mbps is allocated, which is sufficient to transport the 
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high priority traffic without significant queuing delays. The link utilization of the 
high priority traffic is then given by 76%. The aggregate sequence is then filtered by 
the same moving average LPF with 70ms filtering delay. The simulation study in Fig. 
11 shows q^^^ = l,600cells at the total utilization p=96%. The ABR transmission 
5 rates behave similarly to what is already observed in the previous subsections. Note 
that the maximum queuing delay in this case is 10ms whereas it is equal to 40ms in 
the previous single source case. 

Another example has been conducted with the aggregation of ten independent 
2 minute MPEG video sequences from the same movie Star Wars, The average high 

10 priority traffic is 28 Mbps and the link capacity is assigned at C = 60 Mbps. While 
the link utilization of the high priority traffic is 46%, the total link utilization reaches 
93% at q^^^^^ ^ 830 as shown in Fig. 12. Again, the relative large queue length 
behavior in Figs. 1 1 and 12 are mainly caused by the 70ms filtering delay. 
4.6 On Control Stability and Traffic Oscillation 

15 As described above, feedback control schemes which do not consider round 

trip delays may induce low frequency high magnitude oscillations in ABR 
connections. To illustrate the side effect of such oscillations, the load factor approach 
of the OSU scheme for ABR traffic adaptations is examined. As in JAIN et al., the 
load factor is defined as the ratio of the present aggregate ABR arrival rate to its 

20 desired rate, where the desired rate is defined as the remaining link capacity after the 
transmission of high priority traffic. The load factor is computed at 7ms intervals. 
The next ABR rate is equal to its present rate divided by the load factor. In the 
subsequent discussion, refer to this method as the load factor approach. To avoid 
queue congestions, set the desired link utilization p at 80% rather than 100% as in the 

25 previous subsections. In other words, 20% of the total link capacity is kept unused 
when computing the load factor. For comparison purposes, take the same two 
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examples as in Figs. 11 and 12 with the aggregation of ten JPEG/MPEG video 
sources. The same link capacities are assigned here, but no filtering operation is 
implemented on the high priority traffic. 

In the case of the aggi egate JPEG traffic, the simulation study of the load 
5 factor approach shows that the link utilization reaches p=90%, rather than 80% as 
desired, while the maximum queue length is q^^x ^ 9,500 cells as shown in Fig. 13. 
In contrast, using the GPC method provides the solution q^^^^ = 1,600 cells at p=96% 
in Fig. 11. More importantly, the control stability of the load factor approach is 
shown to be highly sensitive to the unpredicted behavior of the high priority traffic. 

10 For instance, when the same load factor approach is applied in the aggregate MPEG 
traffic case, the queuing solution soon becomes unstabilized as found in Fig. 14. 
Comparatively, using the GPC method provides the solution q^^^ = 830 cells at 
p=94% as shown in Fig. 12, which is close to that of the JPEG case. 

Since the CBR/VBR traffic in high speed networks is expected to be highly 

15 bursty and unpredictable, it is of paramount importance for the stability of the ABR 
control scheme to be insensitive and robust to a wide range of CBRA^BR traffic 
characteristics. In the control model design of the present invention, because the high 
priority traffic has been treated as the extemal disturbance, the ABR control stability 
is independent of the high priority traffic. This should be one of the most desirable 

20 ABR control properties. 

In order to further isolate the effect of round trip delays from that of the high 
priority traffic, consider a case without high priority traffic. The link capacity is fixed 
at C=6Mbps for transport of ten ABR connection groups. The aggregate ABR rate 
is set at zero initially. In the ideal control case, the aggregate ABR rate shall soon 

25 reach the total link capacity and stay there ever since. This is exactly what is achieved 
by the GPC method as illustrated by the solid line in Fig. 15, where the ABR rate is 
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normalized by the link capacity C. In contrast, the strong low frequency high 
magnitude oscillations of the aggregate ABR rate is seen when using the load factor 
approach (illustrated by the dotted line in Fig. 15). The oscillation frequency is 
directly associated with the round trip delays; the oscillation magnitude can be as high 
5 as 2.5C. It is obvious that the ABR rate will never converge to C. The oscillation 
behavior is expected to get even worse in the presence of high priority traffic. This 
is well explained in Fig. 16 where the ABR transmission rate in one of the ten groups 
is plotted using the load factor approach. One JPEG video source has been used to 
represent the high priority traffic. The corresponding solution by the GPC method 
r,^^ 10 was previously studied in Fig. 7. 

i! 5 . SIMULATION STUDY OF EFCI-ECD CONTROL SCHEME 

^ The new EFCI-ECD scheme of the invention replaces the queue threshold 

vO detection of the prior art with the bandwidth threshold detection. All the end to end 

i;ri feedback control protocol functions remain unchanged as the existing ones. In most 

n 15 simulation examples of this study, more than 60% buffer capacity can be saved as 
ry compared to the existing EFCI scheme for the same link utilization. Note that the 

traffic filtering operation can easily by implemented by digital signal processing 

-try 

^'"^ (DSP) chips. Using today's technology, a common DSP chip only costs a few U.S. 

dollars whereas the high speed SRAM chips for buffer capacity are relatively much 
20 more expensive. Moreover, a single DSP chip can be shared by many links for 
multiple traffic measurement purposes. 

Computer simulation is used to study the ABR performance. As shovm in Fig. 
18, a single bottleneck link 20 with ten active ABR connection groups is considered. 
The underlying CBRA^BR traffic with high priority transmission are represented by 
25 the superposition of different MPEG video segments collected from the movies Star 
Wars, which are highly bursty and strongly correlated (see, e.g., GARRETT et al.). 
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Due to the delay time constraint on real time services, it is assumed that the 
CBRA^BR traffic alone cannot cause the network congestion, which should be 
ensured by a proper design of call admission control. In the simulation design, the 
link utilization of CBRA^BR traffic is always set at 33%. The purpose of the ABR 
control scheme is to effectively adjust the transmission rate of the ten ABR 
connection groups for a high utilization of the remaining link capacity. 

Let the source destination pair of each ABR connection be denoted by (Si,Di), 
(S|o,Dio), respectively. The round-trip delay for the (Sj^DJ connection is assumed 
to be fixed and equal to 7 kms for k=l, 10, such that the round-trip delay of 
different connections ranges from 7ms to 70ms. Moreover, all sources are assumed 
to be located right at the front of the link, such that there is no delay between the 
sources and the link. Resource Management (RM) cells are periodically generated by 
the sources at every AT==7ms. The original MPEG video files were recorded in bytes 
in 42ms frame time units. The bytes are converted to cells; each cell contams 48 bytes 
of video and 5 bytes of header. In the computer simulation, the queuing process is 
evolved at every 1 .4ms in discrete time. To convert the input fi-ame time unit of 42ms 
into the simulation unit of 1 .4ms, all cells generated in each frame time unit are 
assumed to be randomly allocated among the corresponding simulation time units by 
uniform distribution. 

Throughout the application, two CBRA^BR traffic streams are considered. The 
first one is taken by the superposition of 10 randomly selected 2 minute MPEG 
segments to represent the CBR/VBR traffic on a low speed link. The second one is 
the superposition of 100 randomly selected 2 minute MPEG segments to represent the 
CBRA/^BR traffic on a high speed link. By the law of large numbers, the latter traffic 
is much smoother than the former one. For the 10 MPEG video traffic, the aggregate 
average rate is equal to 3.9(Mbps); the overall link speed is set at C = 1 1.8(Mbps) for 
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the link utilization Pmpeg ~ 

33%. For the 100 MPEG video traffic, the aggregate 
average rate is equal to 38(Mbps); the overall link speed is set at C = 1 12.6(Mbps) for 
Pmpeg = 33%. 

The EFCI scheme is first applied to the above single-node network. The 
5 control parameters Additive Increase Rate (AIR), Rate Decrease Factor (RDF), and 
the queue threshold are properly adjusted to achieve the target overall link 
utilization p=90%. For the 10 MPEG case, (AIR, RDF) = (0.2 Mbps, 0.5) for = 
200 cells. For the 100 MPEG case, (AIR, RDF) - (1 .8 Mbps, 0.5) for = 2000 cells. 
In both cases, Pmpeg = 0.33 and p^gR ^ 0-56. Illustrated in Fig. 19(a) and Fig. 20(a) 

10 are the resuhing buffer occupancy behavior of a 2 minute period. In the 10 MPEG 
case, the average queue is 550 and the maximum is 1,800. In the 100 MPEG case, the 
average queue is 4,500 and the maximum is 14,000. The sample path used in Fig. 17 
is taken from this simulation of the 10 MPEG case. 

Now apply the EFCI-ECD scheme of the present invention to the same 

15 network. The values of the source control parameters AIR and RDF are kept 
unchanged as used for the EFCI scheme. The low pass filter, which is required to 
obtain the filtered CBRA^BR rate, is designed by a simple 100 step moving average 
operation with the time step unit 1 .4ms. Note that the low pass filtering operation at 
the node requires some filtering delay in CBRA/'BR traffic measurement. Such a 

20 filtering delay was considered above. 

To achieve the same target utilization p=90%, the bandwidth threshold is 
simply set at 90% of the link capacity in both 10 MPEG and 100 MPEG cases. The 
resulting buffer occupancy behavior are shown in Fig. 19(b) and Fig. 20(b). For the 
10 MPEG case, the average and maximum queues are found equal to (115, 700), as 

25 reduced from (500, 1 800) of the EFCI scheme. Similarly for the 100 MPEG case, the 
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average and maximum queues are (830, 4000), as reduced from (4500, 14000) of the 
EFCI scheme. 

The aggregate ABR traffic of both schemes for the one second period from 
t=99sec to t^lOOsec are shown in Fig. 21. It is easy to see that the ABR oscillation 
5 ampUtude of the EFCI-ECD scheme is about half of the EFCI scheme. This explains 
why the queuing performance of the EFCI-ECD of the invention is significantly better 
than that of the EFCI scheme. Since the source control parameters in both schemes 
are identical, the EFCI-ECD improvement is mainly achieved through the much 
accelerated ABR traffic response. As seen in Fig. 21, the oscillation period of the 

10 EFCI-ECD is only about one half the oscillation period of the EFCI. As described 
above with reference to Fig. 17, this is attributed to the much reduced congestion 
periods achieved by the bandwidth threshold detection. 

The above comparison is made for a given control parameter AIR with a fixed 
queue threshold Q^. Now examine the effect of AIR and Qt on performance. Taking 

15 the 10 MPEG case, first fix (AIR, RDF) at (0.2 Mbps, 0,5) and adjust G [5, 200]. 
The corresponding average and maximum queuing performance of the EFCI scheme 
is plotted in Fig. 22. Note that every adjustment of will yield a different ABR link 
utilization p^gj^. The queuing performance in Fig. 22 is replotted in Fig. 23 as a 
function of p^gR- For comparison purposes, also in Fig. 23 the average and maximum 

20 queuing performance of the EFCI-ECD scheme as a function of the same p^br is 
shown. In the design of the EFCI-ECD scheme, the bandwidth threshold is assigned 
by P=Pabr"^Pmphg each given p^Qj^. The same degree of 60% buffer capacity saving 
is achieved by the EFCI-ECD scheme for different Pabr s (so as for different Q^'s of 
the EFCI scheme). 

25 Similarly, (Q^, RDF) can be fixed at (200, 0.5) while AIR G [0.2Mbps, 2Mbps] 

is adjusted. The maximum and average queuing solutions of the two schemes are 
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compared in Fig. 24. At each given AIR, first p^br of the EFCI scheme is identified. 
The same AIR and p^^^ are then used in the design of the EFCI-ECD scheme for the 
comparison. In other words, the queue comparison in Fig. 24 is based on equal link 
utilization for the two schemes. Again, about the same degree of buffer capacity 
5 savings has been achieved. Note that RDF is not a sensitive parameter for both 
schemes as long as RDF < 0.5. Its effect on the queuing performance comparison has 
been omitted. 

As compared to the existing queue threshold scheme, the bandwidth threshold 
scheme of the present invention has the general effect of alleviating ABR traffic 

10 oscillations, resulting in a significant saving of buffer resources. 
6. CONCLUSION 

The present invention comprises an explicit rate ABR feedback control scheme 
where the effects of multiloop delays and underlying high priority traffic are fully 
considered. Two distinct techniques are used to significantly improve the ABR 

15 control performance. First, the ABR traffic in the dynamic model design is only 
adapted to the low frequency variation of high priority traffic. As a result, not only 
is the variation of the ABR traffic much smoother, but the control stability is 
remarkably improved. Secondly, the GPC control method has been successfiiUy 
applied to eliminate the low frequency, high magnitude ABR traffic oscillations. 

20 Compared to the existing ABR control schemes, the control scheme of the present 
invention can achieve a significantly higher link utilization with much less buffer 
space requirement. 

Thus, according to the features and aspects of the present invention, a new 
bandwidth threshold detection scheme is provided to modify the original EFCI 
25 scheme for early congestion detection. More than 60% of buffer resource savings is 
observed in the simulation study using the proposed EFCI-ECD scheme. 
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Although the embodiments of the present invention have been described with 
reference to the accompanying drawings, many modifications and changes may be 
made by those skilled in the art without departing from the scope of the invention. 
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